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DETAILED ACTION 
Drawings 

The drawings are objected to because of numerous errors such as reference 
numbers missing from both the specification and drawing sheets and different reference 
numbers for the same element, i.e. Fig. 1 element 10 is missing from drawing and Fig. 2 
elements S3 and S7 are missing from the specification, etc. The drawings are further 
objected to because Figs. 5 (A-C), 8 (A-D), 9 (A-E), and 10 (A-D) are unintelligible and 
Fig. 9D does not disclose colors of microfacets that correspond to numbers of the 
selected cameras found on page 23 of the specification. Corrected drawing sheets in 
compliance with 37 CFR 1 .121(d) are required in reply to the Office action to avoid 
abandonment of the application. Any amended replacement drawing sheet should 
include all of the figures appearing on the immediate prior version of the sheet, even if 
only one figure is being amended. The figure or figure number of an amended drawing 
should not be labeled as "amended." If a drawing figure is to be canceled, the 
appropriate figure must be removed from the replacement sheet, and where necessary, 
the remaining figures must be renumbered and appropriate changes made to the brief 
description of the several views of the drawings for consistency. Additional replacement 
sheets may be necessary to show the renumbering of the remaining figures. Each 
drawing sheet submitted after the filing date of an application must be labeled in the top 
margin as either "Replacement Sheet" or "New Sheet" pursuant to 37 CFR 1 .121(d). If 
the examiner does not accept the changes, the applicant will be notified and informed of 
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any required corrective action in the next Office action. The objection to the drawings 
will not be held in abeyance. 

Claim Rejections - 35 USC § 101 

35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

Claims 14-19 are rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. 

Referring to claims 14-19, a computer program product configured to store 
program instructions for generating an image from a predetermined view direction in 
association with an object to be rendered using a plurality of first images obtained by 
photographing the object to be rendered from a plurality of different directions, and 
second images that pertains to geometry information of the object to be rendered, on a 
computer system capable of performing the method of claims 8-13, is not claimed as 
embodied in a computer readable media. Data structures not claimed as embodied in 
computer-readable material are descriptive material perse and are not statutory 
because they are not capable of causing functional change in the computer. Such 
claimed data structures do not define any structural and functional interrelationships 
between the data structure and other claimed aspects of the invention, which permit the 
data structure's functionality to be realized. 

Claim Rejections - 35 USC § 103 
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The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 1, 2, 5, 8, 9*14, 15, and 18 are rejected under 35 U.S.C. 103(a) as being 

unpatentable over Suzuki et al. U.S. Patent No. 6573912 in view of Beasley U.S. Patent 

No. 5936626. 

Referring to claim 8, Suzuki et al. teaches an image processing method for 
generating an image from a predetermined view direction association with an object to 
be rendered, comprising: generating a plurality of first images obtained by 
photographing the object be rendered from a plurality of different directions (Figs. 1-4; 
column 1, lines 40-49; column 2, lines 30-37; column 7, lines 26-37 and 50-65; column 
9, lines 58-66), and second images that pertains to geometry information of the object to 
be rendered (Figs. 1-4; column 1, lines 40-49; column 2, lines 37-45); generating a 
geometrical shape model of the object to be rendered on the basis of the second 
images (Figs. 1-4; column 1, lines 40-49; columns 2-3, lines 65-3; column 4, lines 15- 
20; columns 7-8, lines 65-8; column 8, lines 20-50; column 9, lines 32-45; columns 9-10, 
lines 58-10; column 10, lines 40-57); generating a plurality of microfacets used to 
approximate a shape of the geometrical shape model (column 3, lines 1-5, i.e. it is 
understood that microfacets are polygons) and generating a third image by selecting 
texture images from the plurality of first images on the basis of the plurality of 
photographing directions and view direction, and projecting the selected texture images 
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onto the microfacets (column 3, lines 42-52); but does not specifically teach executing a 
billboarding process that rotates the plurality of microfacets to make a predetermined 
angle with a view direction. 

Beasley teaches executing a billboarding process that rotates the plurality of 
microfacets to make a predetermined angle with a view direction (column 5, lines 60-67; 
column 6, lines 62-65; column 7, lines 27-41, i.e. the silhouette images are understood 
to be projected texture images). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the method as recited in claim 8 to include 
executing a billboarding process that rotates the plurality of microfacets to make a 
predetermined angle with a view direction thereby reducing the amount of processing 
time required to model complex objects such as trees, which are commonly displayed 
by the use of billboards as they are difficult to model in 3D and to further allow the 
object to rotate in synch with the user's viewpoint so that it's almost constantly facing 
the user (Beasley; column 2, lines 15-24). 

Referring to claim 1, the rationale for claim 8 is incorporated herein, Suzuki et al., 
as modified by Beasley above, teaches an image processing apparatus comprising a 
memory (Figs. 1(elements 118-120 and 126), 2(elements 208-210 and 220), 3(elements 
308-310 and 320), and 4(elements 408-410 and 420); column 7, lines 26-37), a 
geometrical shape model generation unit (Figs. 1 (element 130), 2(element 224), 
3(element 324), and 4(element 424); column 7, lines 26-37), a microfacet generation 
unit (Figs. 1 (element 130), 2(element 224), 3(element 324), and 4(element 424); column 
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3, lines 1-5), and a texture mapping unit (Figs. 1 (elements 1 18-120 and 126), 
2(elements 208-210 and 220), 3(elements 308-310 and 320), and 4(elements 408-410 
and 420); column 7, lines 26-37) configured to perform the method of claim 8 but does 
not specifically teach a billboarding processing unit. 

Beasley teaches a billboarding processing unit (Fig. 1 (element 111) and column 

4, lines 48-65, i.e. it is understood that the geometry subsystem portion of a graphics 
subsystem comprised of a geometry subsystem, a scan conversion subsystem, a raster 
subsystem, and display subsystem comprises the billboarding processing unit). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the image processing apparatus to include a 
graphics subsystem containing a billboarding processing unit capable of rotating the 
plurality of microfacets to make a predetermined angle with a view direction thereby 
reducing the amount of processing time required to model complex objects such as 
trees, which are commonly displayed by the use of billboards as they are difficult to 
model in 3D and to further allow the object to rotate in synch with the user's viewpoint 
so that it's almost constantly facing the user (Beasley; column 2, lines 15-24). 

Referring to claim 14, the rationale for claims 1 and 8 are incorporated herein, 
Suzu8ki et al., as modified by Beasley above, teaches a computer program product 
configured to store program instructions for performing the method of claim 8 (Beasley; 
column 4, lines 18-35 and 48-65). It is inherent that a computer system capable of 
performing the method of claim 8 is comprised of a computer program product 
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configured to store program instructions for executing the method as described in claim 
8. 

Referring to claim 9, the rationale for claim 8 is incorporated herein, Suzuki et al., 
as modified above, teaches a method according to claim 8, wherein the geometrical 
shape model is a voxel model formed of a plurality of voxels (Figs. 1-4; column 1, lines 
40-49; columns 2-3, lines 65-3; column 4, lines 15-20; columns 7-8, lines 65-8; column 
8, lines 20-50; column 9, lines 32-45; columns 9-10, lines 58-10; column 10, lines 40- 
57), and the microfacets are generated for respective voxels (column 3, lines 1-5, i.e. it 
is understood that microfacets are polygons). 

Referring to claim 2, it recites the elements in claims 1 and 9 and therefore the 
same rejections apply. 

Referring to claim 15, it recites the elements in claims 14 and 9 and therefore the 
same rejections apply. 

Referring to claim 12, the rationale for claim 8 is incorporated herein, Suzuki et 
al., as modified above, teaches a method according to claim 8, further comprising 
selecting at least two first images in ascending order of angle that the view direction and 
the plurality of photographing directions make, and generating an interpolated image on 
the basis of the at least two first images, and wherein in texture mapping, the texture 
images are selected for respective microfacets from the plurality of first images or the 
interpolated image on the basis of the plurality of photographing directions and view 
direction, and the selected texture images are projected onto the microfacets (Suzuki et 
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al.; column 1, lines 45-49; column 6, lines 40-46; column 7, lines 50-59; columns 8-9, 
lines 65-10). 

Referring to claim 5, the rationale for claims 1 and 12 are incorporated herein, 
Suzuki et al., as modified by Beasley above, recites the elements in claims 1 and 12 
and further teaches an interpolated image generation unit (Beasley: Fig. 1 (element 111) 
and column 4, lines 48-65, i.e. it is understood that the geometry subsystem of a 
graphics subsystem comprised of a geometry subsystem, a scan conversion 
subsystem, a raster subsystem, and display subsystem is further comprised of a 
interpolated image generation unit) and therefore the same rejections apply. 

Referring to claim 18, it recites the elements in claims 14 and 12 and therefore 
the same rejections apply. 

Claims 13, and 19 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Suzuki et al. U.S. Patent No. 6573912 in view of Beasley U.S. Patent No. 5936626 
as applied to claims 5, 12, and 18 above, and further in view of Neugebauer, P. J., 
"Geometrical cloning of 3D objects via simultaneous registration of multiple range 
images", Shape Modeling and Applications, 1997. Proceedings, 1997 International 
Conference on 3-6 March 1997 Page(s)130 - 139. 

Referring to claim 13, the rationale for claim 12 is incorporated herein, Suzuki et 
al., as modified by Beasley above, teaches a method according to claim 12 further 
comprising appending geometry information each pixel of the plurality of first images 
and the interpolated image on the basis of the second images (column 8, lines 40-50; 
column 9, lines 33-44; columns 9-10, lines 64-9, i.e. it is understood that voxel 
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calculation entails appending geometry information, i.e. depth information from the 
second images, to each pixel) but does not specifically teach executing a clipping 
process of the plurality of first images on the basis of the geometry information of each 
pixel of each first image and the interpolated image, and a distance from a viewpoint to 
each voxel. 

Neugebauer teaches executing a clipping process of the plurality of first images 
on the basis of the geometry information of each pixel of each first image and the 
interpolated image, and a distance from a viewpoint to each voxel (page 135, section 7 
Visibility criterion, 1 st and 2 nd paragraphs; page 137, section 8.3. Direct rendering; Fig. 
9). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the method as recited in claim 9 to include 
executing a clipping process of the plurality of first images on the basis of the geometry 
information of each pixel of each first image and the interpolated image, and a distance 
from a viewpoint to each voxel thereby eliminating self-occlusion errors and making it 
possible to reconstruct concave and convex objects, and even objects with holes out of 
an arbitrary number of range images (page 130, Introduction, 3 rd paragraph). 

Referring to claim 19, it recites the elements in claims 14 and 18 and therefore 
the same rejections apply. 

Claims 3, 10, and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Suzuki et al. U.S. Patent No. 6573912 in view of Beasley U.S. Patent No. 5936626 
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as applied to claims 1-2, 8-9, and 14-15 above, and further in view of Ogata et al. U.S. 
Patent No. 6313841. 

Referring to claim 10, the rationale for claim 9 is incorporated herein, Suzuki et 
al., as modified by Beasley above, teaches a method according to claim 9, but does not 
specifically teach wherein the step of generating the geometrical shape mode includes 
the step of controlling the number of voxels be generated on the basis of precision of 
the second images. 

Ogata et al. teaches wherein the step of generating the geometrical shape mode 
includes the step of controlling the number of voxels be generated on the basis of 
precision of the second images (Fig. 16; column 3, lines 10-28; column 10, lines 16-24; 
column 11, lines 1-19). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the method as recited in claim 9 to include 
wherein the step of generating the geometrical shape mode includes the step of 
controlling the number of voxels be generated on the basis of precision of the second 
images thereby reducing the expensive computing costs due to processing large 
numbers of voxels (column 1, lines 18-27). 

Referring to claim 3, it recites the elements in claims 1 and 9 and therefore the 
same rejections apply. 

Referring to claim 16, it recites the elements in claims 14 and 9 and therefore the 
same rejections apply. 
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Claims 4, 6, 7, 11, and 17 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Suzuki et al. U.S. Patent No. 6573912 in view of Beasley U.S. Patent 
No. 5936626 as applied to claim 9 above, and further in view of Gannett U.S. Patent 
No. 6118452. 

Referring to claim 1 1 , the rationale for claim 9 is incorporated herein, Suzuki et 
al., as modified above, teaches a method according to claim 9, further comprising 
appending geometry information to each pixel of the plurality of first images on the basis 
of the second images (column 9, lines 33-44; columns 9-10, lines 64-9, i.e. it is 
understood that voxel calculation entails appending geometry information, i.e. depth 
information from the second images, to each pixel), but does not specifically teach 
executing a clipping process of the plurality of first images on the basis of the geometry 
information of each pixel of each first image and a distance from a viewpoint to each 
voxel. 

Gannett teaches executing a clipping process of the plurality of first images on 
the basis of the geometry information of each pixel of each first image and a distance 
from a viewpoint to each voxel (column 7, lines 23-46 and 63-67; column 8, lines 34-38; 
column 9, lines 4-9 and 34-43; column 12, lines 34-51; columns 16-17, lines 55-13). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify the method as recited in claim 9 to include 
executing a clipping process of the plurality of first images on the basis of the geometry 
information of each pixel of each first image and a distance from a viewpoint to each 
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voxel thereby providing significant performance enhancements (Gannett: Abstract; 
columns 9-10, lines 60-13). 

Referring to claim 4, Suzuki et al., as modified by Beasley above, recites the 
elements in claims 1 and 9 and further teaches a clipping processing unit (Beasley: Fig. 
1 (element 111) and column 4, lines 48-65, i.e. it is understood that the raster subsystem 
of a graphics subsystem comprised of a geometry subsystem, a scan conversion 
subsystem, a raster subsystem, and display subsystem is further comprised of a 
clipping processing unit) and therefore the same rejections apply. 

Referring to claim 17, it recites the elements in claims 14 and 9 and therefore the 
same rejections apply. 

Referring to claim 6, it recites the elements in claims 1 , 4 and 5 and therefore the 
same rejections apply. 

Referring to claim 7, the rationale for claim 4 is incorporated herein, Suzuki et al., 
as modified above, teaches an apparatus according to claim 4 wherein the clipping 
process unit comprises graphics hardware, see rationale for claim 4 and therefore the 
same rejections apply. 

Conclusion 

The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

The following patents and are cited to further show the state of the art with 
respect to view-dependent texture mapping and clipping. 

Stewart et al. U. S. Patent No. 5898438 
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Sprague U. S. Patent No. 6018348 

Lichtenbelt et al. U. S. Patent No. 6072497 

Georgiev U. S. Patent No. 6268846 

Suits et al. U. S. Patent No. 6525731 

Matusik et al. U. S. Patent Application No. 2003/0231 173 

Efran et al. U. S. Patent Application No. 2003/0137506 

The following patents and are cited to further show the state of the art with 
respect to Billboarding. 

McDowall et al. U. S. Patent No. 5905499 
Lection et al. U. S. Patent No. 6052123 
Barcena et al. U. S. Patent No. 6163320 
Latham U. S. Patent No. 6184857 
McDowall et al. U. S. Patent No. 6285370 
Barcena et al. U. S. Patent No. 6445395 

The following Non-Patent Literature is cited to further show the state of the art 
with respect to view-dependent texture mapping. 

Lifeng Wang; Sing Bing Kang; Szeliski, R.; Heung-Yeung Shum; "Optimal texture 
map reconstruction from multiple views", Computer Vision and Pattern Recognition, 
2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on 
Volume 1, 2001 Page(s):l-347 - 1-354 vol.1 
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Debevec, P.E.; Taylor, C.J.; Malik, J.; Levin, G.; Borshukov, G.; Yu, Y.; "Image- 
based modeling and rendering of architecture with interactive photogrammetry and 
view-dependent texture mapping", Circuits and Systems, 1998. ISCAS '98. Proceedings 
of the 1998 IEEE International Symposium on Volume 5, 31 May-3 June 1998 
Page(s):514- 517 vol.5 

Raskar, R.; Kok-Lim Low; "Blending multiple views", Computer Graphics and 
Applications, 2002. Proceedings. 10th Pacific Conference on 9-11 Oct. 2002 
Page(s)145- 153 

Rademacher, P. 1999. "View-dependent geometry". In Proceedings of the 26th 
Annual Conference on Computer Graphics and interactive Techniques International 
Conference on Computer Graphics and Interactive Techniques. ACM Press/Addison- 
Wesley Publishing Co., New York, NY, 439-446 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Roberta Prendergast whose telephone number is (571) 
272-7647. The examiner can normally be reached on M-F 7:00-4:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ulka Chauhan can be reached on (571) 272-7782. The fax phone number 
for the organization where this application or proceeding is assigned is (571) 273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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